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ABSTRACT 

In a study conducted at a large public university, the authors assessed, for an upper-division 
quantitative methods business core course, the impact of delivery method (online versus face-to- 
face) on the success rate (percentage of enrolled students earning a grade of A, B, or C in the 
course). The success rate of the 161 online students was 55.3%, significantly lower (p = .000) 
than that (72.6%) of the 212 face-to-face students. Both students with a strong (A or B) grade in 
the lower-division statistics prerequisite and students with a weak (C or D) grade in the 
prerequisite had a significantly lower (by approximately 17 percentage points) success rate under 
online delivery than under face-to-face delivery. The study is contrasted with prior studies on the 
relative effectiveness of different modes of course delivery. Implications of the study’s findings are 
discussed. 
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INTRODUCTION 



he eighth annual Sloan Consortium report on online education at degree-granting U.S. colleges and 
universities estimates that 5.6 million higher education students (29.3% of the student body) were 
enrolled in at least one online course in the fall of 2009, as compared to 1.6 million (9.6% of the 
student body) in the fall of 2002 (Allen and Seaman 2010). With increased demand for and reliance on online 
courses, increased importance is attached to ensuring that the convenience and access that online courses afford 
students are not being sought at the expense of student learning and progression in their programs of study. 


This paper shares a study initiated to address two questions about an upper-division quantitative methods 
business course offered online and face-to-face at a large public university: Do students have appreciably greater 
difficulty with the course when they opt to take it online instead of face-to-face? Given that demand for online 
sections of the course exceeds supply, is it advisable to restrict enrollment in the online sections to students who had 
earned a strong (A or B) grade in the lower-division statistics prerequisite? The questions are addressed by 
assessing—for each of students with a strong prerequisite grade and students with a weak prerequisite grade—the 
impact of course delivery method (online versus face-to-face) on the success rate, defined as the percentage of 
enrolled students earning a grade of A, B, or C in the course (as opposed to a D, F, WF, or W). Below, a description 
of the study is preceded by a literature review demonstrating that the study—through its use of the success rate 
metric and its consideration of a student’s prerequisite performance—contributes to the literature on the relative 
effectiveness of different modes of course delivery. 


LITERATURE REVIEW 


This review highlights nine prior studies comparing student performance in an undergraduate business 
course under online and face-to-face delivery. The studies reflect a variety of approaches to making that 
comparison, though none of the studies used or referenced a measure comparable to the success rate. The three 
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studies (Farinella, 2007; Bennett et al, 2007; Trawick, Lile, and Howsen, 2010) that considered a student’s prior 
academic performance did so by focusing on a student’s overall GPA. None of the studies took into consideration a 
student’s prerequisite-specific performance. 

Quantitative Methods Courses 

McLaren (2004) assessed student outcomes in seven online and four face-to-face sections of a statistics 
course (the second in a sequence of two required of all business majors), with all sections taught by her over five 
semesters. Each student was classified by persistence behavior into one of three categories: dropped the course; 
completed the course; or vanished (thereby receiving an F). A very strong relationship between delivery method and 
persistence behavior was evidenced (chi-square = 51.7, p = .000); of the 139 face-to-face students, 5.8% dropped 
and 2.9% vanished, whereas, of the 152 online students, 33.6% dropped and 13.2% vanished. For those students 
completing the course, no relationship between delivery method and course letter grade was evidenced (chi-square = 
2.65, p = .915). 

Lawrence and Singhania (2004) assessed student outcomes in six online and five face-to-face sections of an 
introductory business statistics course, with all sections taught by the same professor over a two-year period. The 
authors found that the mean test performance of the online students was significantly different (p = .0036) and 6 
points lower than that of the face-to-face students. In addition, the percentage of students receiving a W or WU was 
significantly different (p = .024) and 19.5 percentage points higher under online delivery than under face-to-face 
delivery. 

Introductory Finance Course 

Farinella (2007) compared the final exam scores of 33 students in two online sections of an undergraduate 
introductory finance course to those of 103 students in two face-to-face sections of the course, with all the sections 
taught by the same professor. (The mean final exam scores of the online and face-to-face students were 46.97 and 
64.26, respectively.) Controlling (within a regression) for GPA, age, and participation grade, he found a highly 
significant (p = .0001, two-tailed) effect for delivery method, estimating a 21-point lower mean final exam score 
under online delivery than under face-to-face delivery. The coefficient (16.48) of the GPA variable was highly 
significant (p = .0001), indicating a positive ceteris paribus effect for GPA on final exam score. No mention of 
student withdrawals was made. 

Cost Accounting Course 

Chen, Jones, and Moreland (2010) measured, for each of 18 topic areas in an intermediate cost accounting 
course, and based on exam questions, the knowledge level of students in three online sections (n = 75) and three 
face-to-face sections (n = 64) of the course, with all sections taught by the same instructor. For each of the 18 topic 
areas, the mean score of the online students was compared to that of the face-to-face students. Significant 
differences were found for four of the topic areas, with the online students underperforming the face-to-face students 
in three of those four areas. No mention of student withdrawals was made. 

Principles of Economics Courses 

Bennett et al (2007) assessed, for each of an undergraduate principles of microeconomics (micro) course 
and principles of macroeconomics (macro) course, the impact of course delivery method (online versus face-to-face) 
on student numerical course grade. All sections of both courses were offered at the same university during the same 
year and taught by one of three professors, with a total of 240 students completing the micro course and 258 students 
completing the macro course. For the micro course, after controlling (within a regression) for student GPA, ACT 
score, age, and gender, whether or not the student had taken a previous economics course, and professor, the authors 
found that students completing the course under online delivery significantly (p = .05, one-tailed) underperformed 
(by an estimated average of 6 points) students completing the course under face-to-face delivery. For the macro 
course, after controlling for the aforementioned factors, the authors found no significant effect for course delivery 
method. For both the micro and macro courses, GPA (with respective coefficients of 21.22 and 17.66) had a highly 
significant ceteris paribus positive effect on numerical course grade. No mention of student withdrawals was made. 
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Trawick, Lile, and Howsen (2010) compared the exam performance (average score on four exams) of 31 
online students and 78 face-to-face students in sections of a principles of macroeconomics course taught by the same 
professor. Controlling (within a regression) for gender, race, age, ACT score, whether or not the student had 
previously taken an economics class, GPA, credit hour load, hours worked per week, cumulative credit hours 
earned, and whether or not at least one parent had a college education, delivery method had an insignificant impact 
on performance. GPA (with a coefficient of 6.76) had a significant ceteris paribus positive effect on exam 
performance. Based on a subsequent analysis to control for potential self-selection bias in the aforementioned 
regression, the authors estimated a statistically significant 6% lower mean exam performance with online than with 
face-to-face delivery. No mention of student withdrawals was made. 

Principles of Marketing Course 

Weber and Lennon (2007) report on two studies of the same design conducted at separate private 
universities. Each study compared, for a Principles of Marketing course, the performance of students in one online 
section to that of students in one face-to-face section, with both sections having the same instructor, exams, projects, 
and assignments. In each study, for each of mean final exam grade, mean final project grade, and mean course grade 
points, no significant difference was found between the online and face-to-face sections. The authors related that 
"the drop rate was higher for the online sections” (p. 12), but provided no specific withdrawal rates. 

Management Courses 

Friday et al (2006) assessed, for each of two undergraduate management courses (Organization and 
Management, and Strategic Management), the impact of course delivery method and gender on course grade (coded 
on a 12, for A, to 1, for F, scale). The same professor taught concurrent online and face-to-face sections of both 
courses during each of eight consecutive terms. Controlling (within an ANCOVA) for the term the course was taken 
and the professor’s years of experience in teaching the course by the delivery method, the course delivery method 
had no significant effect on course grade for either course, though significant gender effects (with females 
outperforming males) were found for both courses. No mention of student withdrawals was made. 

Schultz, Schultz, and Sieland (2010) analyzed the course grade points of students in online sections (98 
males and 107 females) and face-to-face sections (160 males and 214 females) of Management Fundamentals taught 
by the same professor over the years 2004-2009, with (by university policy) exams proctored for all the students. No 
significant difference in course grade point average was evidenced, for either gender, between online and face-to- 
face course delivery. Also, no significant difference in course grade point average was evidenced, for either delivery 
method, between males and females. No mention of student withdrawals was made. 

STUDY 


The study was conducted at a large public university where the majority of the students work and commute 
to campus. The university defines an online course as one for which at least 95% of the course content is delivered 
over the Internet. No course may be offered online at the university unless it has been judged by a team of faculty to 
meet online learning standards (Quality Matters 2009) of the Quality Matters Program initiated at the University of 
Maryland. The business school offers—for each of its undergraduate business core courses—a small number of 
online sections (typically one or two per term) to supplement its face-to-face sections. 

All undergraduate business majors are required to take two core courses in quantitative methods (hereafter 
labeled QM 2000 and QM 3000). QM 2000 is an introductory business statistics course; QM 3000 addresses linear 
programming, decision analysis, comparisons of population means, and linear regression analysis. Only one online 
section (with enrollment capped at 40 students) of QM 3000 is offered each term and the section fills quickly 
notwithstanding an online tuition surcharge. Apart from that surcharge, online and face-to-face sections of QM 
3000 have the identical entrance requirements, which include QM 2000 as a prerequisite. 
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The subjects in the study were 373 students enrolled in eight sections of QM 3000 taught by the same 
instructor over four consecutive fall and spring terms. For each of those terms, the instructor taught one online 
section and one face-to-face section of the course. (Prior to the study, the instructor had taught QM 3000 online for 
two terms and face-to-face for five terms.) Lectures in both the online and face-to-face sections were organized 
around Power Point slides, with the instructor’s accompanying narration recorded for the online sections and 
provided live in the classroom for the face-to-face sections. The instructor provided the online students a set of 
exercises with answers for practice; in the face-to-face sections, the instructor worked exercises (from the 
aforementioned set) on the board followed, for most class periods, by the students working an additional exercise 
(from the aforementioned set) at their seats for reinforcement. 

For each student, 30% of the course grade was based on non-proctored homework assignments and the 
remainder on two proctored tests and a proctored final exam; the set of graded material was essentially equivalent 
across the four online and four face-to-face sections. The homework assignments were due at approximately two- 
week intervals for both the online and face-to-face students. All students had both on-campus and online access to 
student tutoring in QM 3000 and could phone or visit the instructor during his weekly on-campus office hours. The 
online students (but not the face-to-face students) could additionally ping the instructor’s computer during his office 
hours to initiate an online chat. 

Research Hypotheses 

With the success rate in QM 3000 defined as the percentage of enrolled students earning a grade of A, B, or 
C, and with a student classified as strong if his grade in QM 2000 was A or B and weak if his grade in QM 2000 was 
C or D, the following research hypotheses were posed: 

Hypothesis 1. For strong students, the success rate in QM 3000 is lower under online delivery than under face- 
to-face delivery. 

Hypothesis 2. For weak students, the success rate in QM 3000 is lower under online delivery than under face-to- 
face delivery. 

The hypotheses are consistent with the collective findings of those studies (McLaren, 2004; Lawrence and 
Singhania, 2004; Farinella, 2007; Chen, Jones, and Moreland, 2010) highlighted in the literature review that 
pertained to quantitatively-oriented courses. 

Results and Discussion 

The grades in QM 3000 and (for classification of students as strong or weak) in QM 2000 for the 373 
students were extracted from the university’s student record system. Table 1 provides the distribution of grades in 
QM 3000 and associated success rate under each of online and face-to-face delivery: across all the students, for the 
strong students, and for the weak students. 


Table 1: Distribution of grades in QM 3000 under online and face-to-face delivery 


Delivery 

Students 

A 

B 

C 

D 

F 

WF 

w 

Total 

Success rate 

Online 

All 

38 

25 

26 

15 

21 

1 

35 

161 

55.3% 


Strong 

32 

17 

18 

8 

12 

0 

20 

107 

62.6% 


Weak 

6 

8 

8 

7 

9 

1 

15 

54 

40.7% 

Face-to-face 

All 

63 

55 

36 

22 

14 

1 

21 

212 

72.6% 


Strong 

50 

41 

24 

11 

4 

1 

13 

144 

79.9% 


Weak 

13 

14 

12 

11 

10 

0 

8 

68 

57.4% 


Note: Success rate = percentage of enrolled students earning a grade of A, B, or C in QM 3000. 
Strong = earned grade of A or B in QM 2000. Weak = earned grade of C or D in QM 2000. 
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The success rate was significantly lower (by approximately 17 percentage points) under online delivery than under 
face-to-face delivery: across all students (z = -3.249, p = .000), for the strong students (z = -3.03, p =.001), and for 
the weak students (z = -1.82, .034). Thus (from the latter two significant findings), the two research hypotheses 
were supported. Of note, the strong students realized a significantly higher (by approximately 22 percentage points) 
success rate than did the weak students: under online delivery (z = 2.64, p = .004), and under face-to-face delivery 
(z = 3.43, p = .000). 

Because a student must complete QM 3000 with a grade of C or higher to receive program credit for the 
course, 44.7% of the online students failed to receive program credit for the course, as compared to 27.4% of the 
face-to-face students. The 17 percentage point disparity between the success rates under online and face-to-face 
delivery—observed for strong and weak students alike—implies that students overall do have appreciably greater 
difficulty with QM 3000 when they opt for online over face-to-face delivery. 

The results suggest that face-to-face students are receiving a cumulative benefit from the following features 
unique to the face-to-face sections: students can get immediate answers to questions that arise when new material is 
being introduced in the classroom; the regular twice-weekly classroom meeting schedule imposes a twice-weekly 
pacing in learning the material; and the time spent in class having students work practice exercises at their seats 
ensures immediate reinforcement on the material covered that day. Those features are expected to be of particular 
benefit to students in a highly quantitative course. 

As the strong and the weak students had the same decline in success rate under online relative to face-to- 
face delivery, restricting enrollment in the online sections to strong students would not be anticipated to improve the 
overall percentage of students receiving program credit for the course. Rather, consideration should be given to 
periodically communicating to business majors the success rates (over, say, the past four terms) in QM 3000 under 
each of online and face-to-face delivery so as to inform their decisions as to: whether or not to enroll in an online 
section of the course; and (regardless of their chosen mode of course delivery) how often to avail themselves of the 
opportunity to get assistance from their professor and/or student tutors. 

CONCLUSION 

In a study involving 373 students of a large public university, the success rate (percentage of enrolled 
students earning an A, B, or C) in an upper-division quantitative business core course was found to be significantly 
lower—by 17 percentage points—under online delivery than under face-to-face delivery, both for students with a 
strong (A or B) grade in the prerequisite statistics course and for students with a weak (C or D) grade in the 
prerequisite. The success rate is offered as a fundamental metric—readily derived from centralized student 
records—whereby a business (or other) school can identify those of its courses (if any) for which, at present, 
students are appreciably disadvantaged—from a learning and progression through program of study standpoint— by 
opting for online (or hybrid) delivery over face-to-face delivery. Should one or more courses be so identified, an 
institutional response would be warranted, though deciding upon an appropriate response must take into account the 
university’s mission, policies, practices, and resources. 
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